🤖 Optimize integration test parallelization (maxWorkers 200% → 100%) #443
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ammar-agent
force-pushed
the
investigate-speed-2
branch
from
1e987e0 to
5b2a5d1
Compare
ammar-agent
changed the title
Reduce Jest workers from 200% to 100% on 32-core depot runners. Problem: 64 parallel workers (200% × 32 cores) caused resource contention. Each test spawns child processes, performs I/O, and git operations. Solution: 32 workers (100%) provides better resource utilization and less context switching overhead. Results from testing across 3 CI runs: - Baseline: 193s test execution - Optimized: 173-183s test execution - Average improvement: ~15s (7.8% faster) Note: --silent flag is intentional to prevent overwhelming log output from 32 parallel workers across 17 test files. Failures still show full details. Generated with `cmux`
ammar-agent
force-pushed
the
investigate-speed-2
branch
from
5b2a5d1 to
73c9619
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Optimize integration test parallelization by reducing Jest workers from 200% to 100%, resulting in ~15 second speedup (7.8% faster).
Problem
Running with
maxWorkers=200%on 32-core runners creates 64 parallel workers. Each test spawns child processes, performs I/O operations, and runs git commands. This over-parallelization causes resource contention and context switching overhead.Solution
Reduce to
maxWorkers=100%(32 workers) for better resource utilization.Results
Tested across 3 CI runs:
Note
The
--silentflag is intentionally retained to prevent overwhelming log output from 32 parallel workers across 17 test files. Silent mode still shows test suite status, timing, coverage summary, and full failure details.Generated with
cmux